\title{CS531 Programming Assignment 1: Vacuum-Cleaning Agents}
\author{
        Michael Lam, Xu Hu \\
        EECS, Oregon State University\\
        %\email{}
        %\and
}

\documentclass[12pt]{article}
\usepackage[english]{babel}
\usepackage{graphicx}
\usepackage{subfig}
\usepackage{amsmath}
\usepackage{hyperref}
\hypersetup{
    colorlinks,%
    citecolor=green,%
    filecolor=magenta,%
    linkcolor=red,%
    urlcolor=cyan
}

%\underset{x}{\operatorname{argmax}}
%\underset{x}{\operatorname{argmin}}
\DeclareMathOperator*{\argmin}{arg\,min}
\DeclareMathOperator*{\argmax}{arg\,max}

\begin{document}
\maketitle

\begin{abstract}
In this assignment we design and implement 3 different vacuum-cleaning agents and ran experiments to compare and discuss their performance.
\end{abstract}

% -------------------------------------------------
\section{Introduction}

Consider an environment where the world is a rectangular room consisting of $ n \times m $ cells, with each cell having a probability of dirt. There are three vacuum cleaner agents in consideration: 1) memoryless deterministic reflex agent, 2) randomized reflex agent, and 3) deterministic model-based reflex agent with 3 bits of memory. The agent has three percepts (wall, dirt, home) and five actions (forward, right, left, suck, turn off). The performance measure is the total number of clean cells as a function of how many action steps were taken. We also would like the agent to return to its home cell and shut off after it cleans the entire room.

Having described the environment, agent and performance measure, the general goal of a successful vacuum cleaner agent is to clean the entire room in a minimal number of steps. While that is the ideal case, two of our agents are not able to satisfy this goal. Thus we have different goals for our agents due to their designs by definition. The first agent in consideration, the memoryless deterministic reflex agent, is limited by pure reflex actions with no memory. We discovered that we cannot clean every cell with this agent, so our goal for this agent is to clean as many cells as possible and to return home at the end. For the randomized reflex agent, its performance is improved and simultaneously hindered by its probabilistic actions. We discovered that while it can clean the entire room over time, it is difficult to maneuver the agent back to the home cell due to randomness. The goal for this agent then is to clean every cell without needing to return home. Finally, the deterministic model-based reflex agent has memory, which gives it advantage to remember which actions it has taken. Given this advantage, the goal of this agent is to clean the entire room and navigate home successfully. We determined that this was possible in less than 8 states, the maximum cap for the agent's memory.

\input{1_reflex_agent}
\input{2_random_agent}
\input{3_model_based_agent}
\input{Experiments}

\section{Discussion}
We have implemented three reflex agents and have shown that simple tasks can be done with carefully designed rules. However, it is not a good idea for creating an intelligent machine which has to handle many complex tasks. A better solution may let it learn how to do a certain type of tasks by itself. Actually, we can learn model parameters or mapping rules for our agents. We will try to use Q-learning for this improvement. But such learning agent is beyond the scope of this project.

\bibliographystyle{plain}
\bibliography{ref}

\end{document}
This is never printed
